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Complex Electronics (CE) now perform tasks that were previously handled in software, 
such as communication protocols. Many methods used to develop software bare a close 
resemblance to CE development. Field Programmable Gate Arrays (FPGAs) can have 
over a million logic gates while system-on-chip (SOC) devices can combine a 
microprocessor, input and output channels, and sometimes an FPGA for programmability. 
With this increased intricacy, the possibility of “software-like” bugs such as incorrect 
design, logic, and unexpected interactions within the logic is great. 

With CE devices obscuring the hardware/software boundary, we propose that mature 
software methodologies may be utilized with slight modifications in the development of 
these devices. Software Process Assurance for Complex Electronics (SPACE) is a 
research project that used standardized S/W Assurance/Engineering practices to provide 
an assurance framework for development activities. Tools such as checklists, best 
practices and techniques were used to detect missing requirements and “bugs” earlier in 
the development cycle creating a development process for CE that was more easily 
maintained, consistent and configurable based on the device used. 
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■ICOrpplex 0fectt:omcs ;(GE‘) 
devices can have over 
one million gates and 
even a built in 
microprocessor. These 
devices are replacing 
conventional Hardware 
and software in many 
critical applications. Row 
do we assure quality on 
tltese devices^ 
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What has been done? 


Software/Hardware techniques have been modified to 
work with complex electronics devices. 

Currently working with multiple projects to verify and 

refine 

• Checklists 

• Techniques 

• Templates 

-Assurance Plan 
-Audit 

Flow diagrams have been created to aid in CE tracking 
Classes have been created for training 






Simple or Complex? 


The Federal Aviation Administration (FAA) provides a definition in 
DO-254, “Design Assurance Guidance for Airborne Electronic 
Hardware” document. It states “A hardware item is identified as 
simple only if a comprehensive combination of deterministic tests 
and analyses appropriate to the design assurance level can ensure 
correct functional performance under all foreseeable operating 
conditions with no anomalous behavior. When an item cannot be 
classified as simple , it should be classified as complex. An item 
constructed entirely from simple items may itself be complex.” 

Firmware is not CE. The most common definition is found in IEEE 
610.12-1990: “The combination of hardware device and computer 
instructions and data that reside as read-only software on that 
device.” 
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•onics exdcutes.safety-critical functions 


lies executes mission-critical functions and is a single point of failure 
id to be highly complex 

;d to have significant risk due to one or more of these factors: 



sign is expected to have significant risk due to one or more of these factors: 

:auir ements 

_„ncems^vith the chosen-technology, suph as power consumption, design size 
for the chip, timing, packaging, or operating frequency 

TT: ~Hy innovative and untried design approach 
ly aggressive schedule 
Inexperience of the design team 


The complex electronics executes mission-critical functions but there is redundancy in the 
system 

The design is expected to be moderately complex 
3. The design is expected to have moderate risk due to one or more of these factors: 

i. Some requirements undefined or unstable 

ii. Somewhat innovative and untried design approach 

iii. Aggressive schedule 

iv. Design team contains some inexperienced members 


The complex electronics is classified as Low if it does not fall into either of the above categories 
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Assurance Process Planning Checklist 


ID 

Process Step 

Completion 

Date 

Eng 

QA 

1 

Entrance Criteria are met 




2 

Determine complex electronics criticality classification (high, 
moderate, or low). 




3 

Generate Complex Electronics Development Plan (Eng.). 




4 

Generate tailored Complex Electronics Assurance Plan. 

(See “Process Checklist for Assurance Planning” for details.) 





Update project plans with complex electronics information or 
create new plans for: 




5 

- Safety 




6 

- Risk Management 




7 

- Configuration Management 




8 

- Problem Reporting and Corrective Action process 




9 

- Verification and Validation 




10 

Exit Criteria are met 
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How do You Start the Process? 


■ Create an Assurance 
Plan for your device 

□ Can be stand alone or 
part of the larger 
assurance plan 

□ Plan is based on 
criticality of device(s) 

□ Get concurrence with the 
plan 



One plan can cover all 
devices needed 
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Assurance Planning Document 




ID Process Step 


Entrance Criteria are met. 


Concur with complex electronics criticality classification (high, moderate, or 
low). Include in the CEAP (below) 


Generate tailored Complex Electronics Assurance Plan (CEAP). This 
includes steps 4 through 14 below. 


Specify purpose and scope of plan. 


Determine Applicable and Reference standards and documents. 


Depie management of the assurance activities, including organization 
structure, roles and responsibilities, resources, schedule, reporting. 


Identify training and tools required for the assurance tasks. 


Specify tasks for Requirements phase, including reviews, audits, and 
analyses. 


1 3 Specify criteria and tasks for acceptance of complex electronics. 


20 Exit Criteria are met. 


Completion QA 
Date 





Richard .A. Plastow (SAIC) 

Sr. Software Assurance Engineer 


9 






















Supporting Processes 

■ Controlling and monitoring the process used 
is very important. 

□ Configuration Management 

□ Problem reporting / monitoring 

□ Continuous Risk Management 

□ Audits 

Ad Hoc is NOT a process! 
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system-level testing 


e requirements for the board or chip level 
H Qesig^leetrmjps^meet the requirements, TTsing good engineering practices 
Bpmplerhent the design m hardware 
Bpest the hardware; Implement changes 


Rldentifyif complex eleetfonics can cqyse a hazard or are part of a hazard control 
BjEnsuretFrat design errors in complex electronics are considered as a failure mode 


S* "Assess entrance and ejdt criteria for each life cycle phase 

IpEnsure tfaceability of the requirements through all levels of development 

•^Analyze the products produced (documents,, designs, etc.) against the requirements 
and the output of the previous phase 
• perform white box analysis on CE designs and tests 
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The first step in any design process should be to define and 
document the requirements end constraints under which the CE 
must operate^This allows you to think through the issues and 
document any design decisions and trade-offs . 

Complex JSectronics desigrf is often 1 started based on the engineers 
||nowledge of the system, not defiled fequirejnents.HBP^ ^ 7 * 

Requirements Reviews 

■ Clear 

■ Concise 

■ Confirmable 

■ Traceable 

frterface Control Docurrrent verifications 
Signals pst 
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Requirements Review Checklist Snippet 


ID 

Topic 

Criteria 

Yes/No/ 

NA 

Comment 

2 

General 

Are the overall system configurations and operating modes 
defined? 



11 

Interfaces 

Is the data size and bit order defined? 



12 

Signals 

For each input signal, is the range of valid data defined? 



13 

Signals 

For each output signal, is the range of valid data defined? Is a 
typical value defined? 



20 

Initial 

Conditions 

During power-up or reset, do any lines or signals float? 



27 

Error 

Handling 

Is the behavior of the CE in response to an external failure or 
fault specified? 



28 

Timing 

Has the timing of critical signals been defined? 
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Design 

One major difference between CE and Software is 
the aspect of timing and concurrency. 

□ Design reviews are important. 

□ Independent engineer should review the design 

□ Confirm the design supports the requirements 

□ Use a coding standard 

□ Have and follow Best Practices 
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Code P llpflementation 


Although HDlJis not true code, it shares many of 
Hie s^etea^ces^d^attri^Erte^^of software. 
Differences include: 



he placement of the logic blocks within the 
chip, and the routing between blocks, are 
some of the processes that occur during 
implementation (link). 
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Ease of Coding 


■ Coding Standards, Code Reviews and Best 
Practices work well on HDLs. They allow: 

□ Readability 

■ Standard Signal names 

■ Names do not change across boundaries 
Common register names 

□ Maintainability 

□ Common naming conventions 
Code reviews 

Etc.... 
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Code Example 


.Ucrarv 

TSH ^ 5 =..==. - 3 td logic 116^ . aCC; 

[ENT mi rsg^ 3 3 FORT ( 

clk: f we: CM 3 td_iogic; 

rdata: IN 3 td logicj/ector (7 D-CWNT O O) 

AesI, EbsI : ITT 3 1 d_Io g ic_vs ct or (1 DOWHTO O) ; 

Aoizi=. r Bonn. : OUT logic vector (7 DDi#TTD O) j ; 

I TUT: o^ecrE ; 


;tue 


rXTT 
rst. : 


FRO! 


OF 


( c^k:. 


:dat( 


■) 


tyf: 


IRAT (O TO 3) OF s t d_Io g ic_ve ct or (7 DOWNTO O) ; 
:ey(7 DOWHTO O i ; 


rZTI 


: F c Ik p EVENT AMD cik='0 
X F l we = p 1 r ]i T H EM 


TEEN 


C A^n ^ A3sl. 

3 



WHEN 

M OO n => 

re^ ( Q) 

: =rdata ; 

WHEN 

rT 01 w => 

ieg (C) 

: =rdata ; 

WHEN 

r, :o n => 

( 2 ;i 

: =rdata „■ 

WHEN 

OTHERS = 

> rec (3) : = 07d.au. a; 

END- CA5E; 




CASE iA3 s i_ 

TS 



WHEN 

M OO n => 

Aoizl=.<= 

-ecr (O) ; 

WHEN 

M Ol ri => 

AOU t=- <= 

-sc- ( 1 . ) ; 

WHEN 

r TO n => 

AOUI t=- <= 

-egr (2 ) ; 

WHEN 

OTHE ERS = 

> AOTJLt 

< = zsg i_ 3 j .r 

END- CASH 

! ; 



CASH B 3 e 

r\_ 75 



WHEN 

r. OO ri => 

3 oi JL H- <= 

-sc- (O) ; 

WHEN 

" Ol n => 

Bofut.<= 

-ecr i-) ; 

WHEN 

TO n => 

BOTJLt< = 

-“O' (2 ) ; 

WHEN 

OTHE R3 = 

> Bo\zt=-<=07&^ (3) ; 


CMC- 


TO 


HUD 

CMD FRGCE; 


[>TD one: 
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Example of Code Review Best Practices 


ID 

Topic 

Criteria 

Yes/No/ 

NA 

Comment 

5 

General 

All states in a state machine are defined or invalid states are 
returned to a known state 



25 

Clocks 

Do not generate the clock using combinational logic 



27 

Signals 

Names shall be self-explanatory 



34 

Reset 

Provides an Asynchronous reset line 



36 

Interfaces 

All asynchronous inputs are first synchronized before use 



39 

Functions 

The sensitivity list contains only the signals that should cause the 
process to be executed 



41 

Other 

Are unused functions within the IP cores identified? Is the accidental 
execution of these functions prevented? 
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Test 


While complex electronics use test benches 
and timing models, the idea of a well defined 
suite of test cases is common in both 
disciplines. This includes test plans, fault 
injection and error handling testing and 
verification. 
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Best Practices 
Test Plans 

> Tracing to requirements 

> Feasible 

. Co^tJmoreiiarT’just success 
■ Fault Injection 
^ T es? VerifcalSon JP 

Proflem Tfend Anatysis / Tracking 
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Test Plan Review Checklist 


ID 

Topic 

Criteria 

Yes/No/NA 

Comment 

4 

General 

Is the functionality of the complex electronics (CE) in off- 
nominal mode being tested? 



14 

Interfaces 

Interfaces with COTS IP modules tested 



17 

Signals 

For each input signal, is the range of valid data tested? 



25 

Initial 

Conditions 

Is the state of programmable memory elements upon power- 
up and after a reset tested? 



27 

Error Handling 

Have all expected errors or faults been tested to verify they are 
handled correctly? 



30 

Tinning 

Has the timing of critical signals been tested? 



32 

Power/ 

Electrical 

Is the maximum allowable power and current to the CE being 
verified? 
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Reality Check 

■ Many assurance engineers, regardless of their 
specialty, have little understanding of the 
complexities of these devices. Any review done will 
only be to the level of knowledge of the assurance 
engineer. 

■ Three courses have been developed 

□ Introduction to CE 

□ CE Life Cycle 

□ Assurance of CE devices 

Techniques and checklists were created to ensure 
quality. 

Not all techniques/checklists used on every part. 
Dependent on criticality and project direction. 
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Process Flow Example 


CERP 


CEDP 


CEDDP 


IMPL 


V &, V 


Acce pta nee 
and Release 


OPS 



■Software- ■Qualify to Hardware Quality 
Transition 

Bi-diractianal Rsqu ii amants »racaabi5ity - 
Software Qualify, Safety and Reliability 
nnmnnnnnT TfreNno - Hardware Quality 
TP iS ■' .Xe^t B I gji ■' .Te^t F; munrese review - 

Sallware Quality. Hale ly and Reliability 

Systam Int&aratlQn Testing - Hardware 
Quality 
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y D ec i s i o n ; Tif W^Ttees 

* Design Evaluation 

* Design Review 


* Failure Mode and Effect Analysis 

* Fault Tree Analysis 

*'gjjnctic5n anjJ Physical Confgurdtion Audits 

* Interface Ifnalysis 

* IRequ ire rife nts ^valuation 

* Meqdir^rnents Review 
■ Risk Analysis 
■Pfraceability Analysis 
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Rational for Change 
Effects on Internal Interfaces 
Effects on External Interfaces 
Effects on Hazards 
Effects on Operations 
Potential for introducing new bugs 
impact* of Change (Minor/Major and why) 
Tesfing/Velifications needed 
Things to Consider 
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Checklists 


■ Planning Phase 

■ Requirements Phase 

■ Preliminary Design Phase 

■ Detailed Design Phase 

■ Implementation Phase 

■ Testing Phase 

■ Operations Phase 

■ Assurance Planning 

■ Modifications or Upgrades 

■ Audits (Functional Configuration, Physical Configuration and 
In-Process) 

j Best Practices (Code Review) 
j Testing (Document Review) 
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Requirements Phase Process Checklist 

■ Overview 

■ Entrance Criteria 

■ Responsible Personnel 

■ Process Step - Lists Analysis to use/update, 
documentation(to create, update, etc.), 
supporting processes needed 

Exit Criteria 

Documentation Status 
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Conclusion 


Thanks to the NASA Software Assurance Research Program 
which funded this Research and the IV&V center for supporting 
the infusion of this research into NASA projects. 
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